Beware of using invalid transmission models to guide HIV health policy.

Jeff rey Eaton and colleagues (January, p e23) used modelling to inform international guidelines for antiretroviral therapy for HIV. Specifi cally, they undertook a comparative cost–effectiveness analysis based on predictions from 12 independent transmission models: seven were used for South Africa, four for Zambia, four for India, and one for Vietnam. They concluded that all 12 models show similar results—that earlier eligibility for antiretroviral therapy is cost eff ective. They implied that their consensus fi nding increases confidence in the use of modelling results to guide HIV health policy. The comparative consensus modelling approach used by Eaton and colleagues is now used frequently to justify the implementation of particular HIV interventions. However, this approach produces meaningful results only if the comparison is based on valid models. Most comparative consensus studies (including that by Eaton and colleagues) have not checked for model validity. We checked the validity of the seven South African models that Eaton and colleagues used by reproducing some of their numerical simulations (fi gure). A valid model should accurately refl ect the current state of the epidemic. Notably, we show that not all of the South African models agree on the present state of the epidemic; consequently, they cannot all be valid. Two models estimate that, at present, the epidemic (ie, incidence) is fairly stable, three that it is gradually decreasing, and two that it is rapidly decreasing (fi gure). One can only have confi dence in the results from a comparative analysis if it is based on valid models. Only then can a comparative consensus approach be useful to guide health policy.